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(54) A range-image-based method and system for automatic sensor planning 



(57) A system 1 0 for measuring one or more dimen- 
sions of a physical part 18 includes a fringe generator 
14 for projecting a fringe pattern of light onto a surface 
of the physical part 1 8 to be measured. The system 1 0 
includes a sensor 1 2 for capturing images of the fringe 



pattern on the surface. The sensor 12 and the fringe 
generator 14 are located at different positions. The 
fringe generator 1 4 and the sensor 1 2 are each in com- 
munication with a computer 16 to determine whether 
each of the fringe generator 1 4 and the sensor 1 2 have 
line-of-sight visibility to the surface to be measured. 
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Description 

[0001] This invention relates to an automatic sensor 
planning system and method and more particularly to a 
range-image based automated sensor planning system 
and method to assist in accurately determining the sur- 
face geometry of a part. 

[0002] Part inspection is an important step in manu- 
facturing and many part inspection techniques are 
known. Recently, automated part dimensional inspec- 
tion techniques have been developed to solve some of 
the problems that are present in traditional approaches, 
including accuracy and speed. An essential part of every 
automated inspection system is finding suitable config- 
urations for the sensors or sensor planning so that the 
inspection tasks can be satisfactorily performed. 
[0003] One prior known system proposed an auto- 
mated dimensional inspection environment for manu- 
factured parts using a Coordinate Measuring Machine 
(CMM). This system utilized CAD databases to gener- 
ate CMM sampling plans for inspecting or analyzing the 
surface of the part. This CMM method was accurate, but 
extremely time consuming as it employed a point-by- 
point sampling system. The method became even more 
time consuming when the system was used to measure 
the surface of large parts. Other traditional point-scan 
devices, such as line-scanning devices and laser scan- 
ners, suffer from the same problems. Moreover, this 
method could only be utilized when a CAD model was 
available. 

[0004] Active optical scanning methods are also 
known for part surface inspection. These methods allow 
for faster dimensional inspection of a part. One current 
active optical sensing method that has been success- 
fully employed for various applications is the structured 
light method, which obtains 3-D coordinates by project- 
ing specific light patterns on the surface of the object to 
be measured. However, sensor configurations, such as 
position, orientation, and optical settings, are critical to 
the structured light method. These configurations effect 
measuring accuracy and efficiency directly. In most prior 
structured light applications, sensor configuration plan- 
ning was based on human operator experience, which 
resulted in considerable human error and thus, low ef- 
ficiency. These methods are also typically not as accu- 
rate as the point-scan methods, which are discussed 
above. 

[0005] Currently, sensorplanning in acomputervision 
environment attempts to understand and quantify the re- 
lationship between the object to be viewed and the sen- 
sor observing it in a model-based task directed way. Re- 
cent advancements in 3-D optical sensor technologies 
now allow for more efficient part inspection. However, 
these sensor technologies are still too inefficient for use 
in most commercial production processes. 
[0006] Presently, the most widely used 3-D method 
for sensor planning for part inspection is the "click and 
check" method. In the click and check method, the user 



is presented with a graphical display of the object to be 
measured based on a CAD model. Based on the CAD 
model, a file is written and then translated into a file that 
a CMM/robotics off-line programming package can 

5 read. The programming package, such as SILMA or 
Robcad is used to develop a program that will move the 
CMM/robot along the predefined path. By using the off- 
line programming package, a user/operator must imag- 
ine the 3-D object in space and then insert locations and 

10 view directions for the sensor by clicking the points in 
the graphical display. Having developed a set of sensor 
locations, each location must be verified to insure that 
it was acceptable and the entire surface is covered. Usu- 
ally, this is done using a physical part and a CMM or a 

15 robot. 

[0007] The click and check method also provides a 
technique for connecting the locations in order to form 
a sensor path. As is known, other technology is em- 
ployed to control how the CMM or the robot moves the 

20 area scanner between locations without collisions or 
kinematic inversion problems. The click and check 
method is extremely time consuming, difficult to per- 
form, and also unreliable. Moreover, because it requires 
human intervention and selection of the view direction 

25 of the scanner for each location, it is susceptible to sig- 
nificant error and, thus, inefficiency. Additionally, this 
method can only be utilized when a CAD model of the 
surface to be measured is available. 
[0008] It is therefore an object of the present invention 

30 to provide a sensor planning system and method that 
automatically determines sensor locations without the 
need of a CAD model. 

[0009] According to a first aspect of the invention 
there is provided a system for measuring at least one 

35 dimension of a physical part characterised in that the 
system comprises a fringe generator for projecting a 
fringe pattern of light onto a surface of the physical part 
to be measured, a sensor for capturing images of said 
fringe pattern on said surface, said sensor being located 

40 at a different position from said fringe generator, a com- 
puter in communication with both said fringe generator 
and said sensor to determine whether each of said 
fringe generator and said sensor have line-of-sight vis- 
ibility to said surface to be measured. 

45 [0010] Said fringe generator may include a light 
source directed at said surface and a spatial light mod- 
ulator disposed between said light source and said sur- 
face. 

[0011] The system may further comprise a phase 
50 shifter for generating incremental phase shifts of said 
fringe pattern on said surface, which incremental phase 
shifts are captured as images by said sensor. 
[0012] Said phase shifter may be digitally controlled 
by said computer. 
55 [001 3] Said computer may determine a combined im- 
age based on said captured images of said incremental 
phase shifts. 

[0014] Said combined image may be determined ac- 
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cording to the equation : 

l = arctan ((l 4 -l 2 ) / (l r l 3 )) 

[0015] According to a second aspect of the invention 
there is provided a method for providing automatic sen- 
sor planning to measure the shape and dimension of a 
physical part, comprising projecting a fringe pattern on 
a surface of the physical part, projecting an incremen- 
tally phase shifted fringe pattern on said surface and de- 
termining based on said projected fringe pattern and 
said projected incrementally phase shifted fringe pattern 
whether a sensor and a fringe generator have a clear 
line-of-sight to said surface. 

[0016] A determination of whether the sensor and 
said fringe generator have a clear line-of-sight may be 
based on an image of the projected fringe pattern and 
said projected incrementally phase shifted fringe pat- 
tern. 

[0017] The method may further comprise computing 
a combined image resulting from an image of the pro- 
jected fringe pattern and an image of the projected in- 
crementally phase shifted pattern. 
[0018] The method may further comprise scanning 
said combined image to determine whether the sensor 
and the fringe generator have a clear line-of-sight, a 
poor line-of-sight, or no line-of-sight. 
[0019] The method may further comprise adjusting a 
position of said sensor if any bars in said combined im- 
age are not continuous. 

[0020] The method may further comprise adjusting a 
position of said fringe generator if a significant spacing 
exists between any two adjacent bars in said combined 
image. 

[0021] The method may further comprise adjusting 
the fringe generator so that a surface areawith poor line- 
of-sight is exposed to more tightly spaced fringe bars 
relative to a surface area with clear line-of-sight. 
[0022] The method may furthercomprise capturing an 
image of said projected fringe pattern and capturing an 
image of said projected incrementally phase shifted 
fringe pattern. 

[0023] The method may further comprise creating a 
ROPD diagram corresponding to said phase shift. 
[0024] According to a third aspect of the invention 
there is provided method for determining whether a sen- 
sor and a fringe generator have line-of-sight visibility to 
an object surface, comprising inputting phase shift im- 
ages to a computer, computing a combined image 
based on said input phase shift images, scanning said 
combined image and determining whether any occlu- 
sions result in no line-of-sight or poor line-of-sight from 
either the sensor or the fringe generator to the object 
surface. 

[0025] The method for determining whether a sensor 
and a fringe generator have line-of-sight visibility may 
further comprise computing an ROPD diagram for any 



of a set of scan lines chosen for said phase shift images. 
[0026] The method for determining whether a sensor 
and a fringe generator have line-of-sight visibility may 
further comprise determining whether any bars in said 

5 combined image are not continuous and determining 
whether any significant variation of spacing exists be- 
tween any bars in said combined image. 
[0027] The method for determining whether a sensor 
and a fringe generator have line-of-sight visibility may 

10 further comprise determining whether any peaks are 
missing in said ROPD diagrams and adjusting the loca- 
tion of the sensor if any peaks are missing in said ROPD 
diagram. 

[0028] The method for determining whether a sensor 
15 and a fringe generator have line-of-sight visibility may 
further comprise determining whether there are any ad- 
jacent pairs of peaks with excessively small interval be- 
tween them and adjusting the location of the sensor if 
there are any adjacent pairs of peaks with an excessive- 
20 |y small interval between them. 

[0029] The method for determining whether a sensor 
and a fringe generator have line-of-sight visibility may 
further comprise determining whether there are any ad- 
jacent pairs of peaks with an excessively large interval 
25 therebetween and adjusting the location of the fringe 
generator if any pairs of peaks exist with an excessively 
large interval therebetween. 

[0030] The method for determining whether a sensor 
and a fringe generator have line-of-sight visibility may 

30 further comprise determining whether there are any ad- 
jacent pairs of peaks with an excessively large interval 
therebetween and decreasing the interval among those 
fringes that are projected onto the area where peaks 
with an excessively large interval exist. 

35 [0031] The method for determining whether a sensor 
and a fringe generator have line-of-sight visibility may 
further comprise determining whether any peaks are 
missing in said ROPD diagram and adjusting the loca- 
tion of both the sensor and the fringe generator if any 

40 peaks are missing in said ROPD diagram, if any signif- 
icant variation of spacing exists between any bars of 
said combined image and if any bar in said combined 
image is not continuous. 

[0032] The method for determining whether a sensor 
45 and a fringe generator have line-of-sight visibility may 
further comprise determining whether any peaks are 
missing in said ROPD diagram and adjusting the spac- 
ing of the fringes so that an area with poor line-of-sight 
from the fringe generator is covered by more tightly 
50 spaced fringe bars. 

[0033] The invention will now be described by way of 
example with reference to the accompanying drawing 
of which:- 

55 Figure 1 is a schematic view illustrating the compo- 
nents of a sensor system in accordance with a pre- 
ferred embodiment of the present invention; 
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Figure 2a is a side view of a pyramid -shaped object 
that is used to exemplarily illustrate a preferred 
method in accordance with the present invention; 

Figure 2b is a front view of the pyramid-shaped ob- 
ject showing Figure 2a; 

Figures 3a through 3d illustrate exemplary fringe 
pattern images captured from the pyramid-shaped 
object of Figures 2a and 2b corresponding to phase 
shifts in accordance with a preferred embodiment 
of the present invention; 

Figure 4 is an illustration of an image, which is a 
combination of the phase shift images of the pyra- 
mid-shaped object of Figures 3a through 3d in ac- 
cordance with a preferred embodiment of the 
present invention; 

Figures 5a through 5d illustrate exemplary fringe 
pattern images captured from a pyramid-shaped 
object where a fringe generator has visibility to the 
object surface but the sensor does not have visibility 
to a portion of the object surface in accordance with 
the present invention; 

Figure 6 is an illustration of an image, which is a 
combination of the phase shift images of the pyra- 
mid-shaped object of the phase shift images of Fig- 
ures 5a through 5d, in accordance with a preferred 
embodiment of the present invention; 

Figures 7a through 7d illustrate exemplary fringe 
pattern images captured from a pyramid-shaped 
object where a sensor has visibility to the object sur- 
face, butthefringe generator does not have visibility 
to a portion of the object surface, in accordance with 
a preferred embodiment of the present invention; 

Figure 8 is an illustration of an image which is a 
combination of the images of the pyramid-shaped 
object of the phase shift images of Figures 7a 
through 7d, in accordance with a preferred embod- 
iment of the present invention; and 

Figure 9 is a schematic flow chart illustrating the de- 
cision flow of an automatic sensor planning system 
in accordance with a preferred embodiment of the 
present invention. 

[0034] Referring now to Figure 1 , which schematically 
illustrates a sensor system 1 0 in accordance with a pre- 
ferred embodiment of the present invention. The sensor 
system 1 0 includes a sensor or camera 12 and a fringe 
generator 14, which are each in communication with a 
computer 16. 

[0035] The camera 12 is preferably a digital camera 
that is in communication with a ferroelectric LCD shuttle 



driver, however, any commercially available camera 
may be utilized, in which case it may be necessary to 
scan and digitize the obtained images. 
[0036] The fringe generator 1 4 is preferably a spatial 
5 light modulator ("SLIM") that projects a fringe pattern of 
light onto a surface of an object 1 8 that is being inspect- 
ed or measured. 

[0037] In accordance with a preferred embodiment, 
the SLM system includes a laser 20, which emits a beam 
10 of light 22 through a liquid crystal spatial light modulator 
24. The laser is preferably a YAG laser in communica- 
tion with an objective lens however, any commercially 
available lens may be utilized. 

[0038] The liquid crystal spatial light modulator 24 di- 
15 vides the beam of light into multiple beams 26 to create 
a fringe pattern on the surface of the object 18 to be 
inspected or measured. The liquid crystal spatial light 
modulator can be digitally controlled. This allows the 
computer to control how the beam of light is divided. For 
20 example, the computer can be programmed to control 
the spacing of the fringes. 

[0039] The fringe generator 1 4 can be any of a variety 
of known fringe generation systems including the fringe 
generator disclosed in U.S. Patent No. 6 : 100,984. 

25 [0040] The camera orsensor 12 captures grey-scaled 
images of the fringe pattern or patterns projected onto 
the surface of the object 18. Preferably, the sensor 12 
and the fringe generator 1 4 are not located at the same 
position in order to ensure accuracy. Preferably, the 

30 fringe generator 14 generates a fringe pattern, com- 
prised of vertical bars. The sensor 1 2 captures a photo- 
graphic image of a grey-scaled image of these bars. The 
fringe pattern can obviously take on a variety of other 
configurations and patterns. However, vertical fringes 

35 are preferred in the preferred embodiment where the 
sensor and fringe generator are positioned relative to 
each other in a horizontal configuration. 
[0041] In order to illustrate more clearly the present 
invention, a pyramid-shaped object is used to assist in 

40 explaining the characteristics of the fringe patterns un- 
der various circumstances related to the present inven- 
tion. 

[0042] Figures 2a and 2b are a side view and a front 
view respectively of the pyramid-shaped object 28, 
45 which is utilized to illustrate the operation of the dis- 
closed method. 

[0043] Referring now to Figures 3a through 3d, which 
illustrate several fringe pattern images captured from 
the exemplary pyramid-shaped object 28. The highest 

50 intensity is along the middle of a white bar and the lowest 
intensity is along the middle of a dark bar, generally in- 
dicated by the shaded portions. Normally, the two peak 
intensities of two neighbouring or adjacent bars repre- 
sent a complete "phase" ; which is 360 degrees, and the 

55 intensity in-between is modulated and sinusoidal in na- 
ture. 

[0044] As is known, an SLM can shift the position of 
these bars through a technique known as phase shifting. 
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A full phase shift, i.e. a 360-degree shift, means each 
bar is moved to the position of its left or right neighbour 
consistently. 

[0045] Usually a fringe pattern generated from an 
SLM is a set of parallel and uniformly spaced light 
bands. Their projections onto an object may become 
non-parallel and vary in width due to the position, orien- 
tation, and shape of the object relative to the fringe gen- 
erator and the sensor. 

[0046] Figure 3a illustrates an image of an initial fringe 
pattern projected onto a pyramid-shaped object 28. Fig- 
ure 3b illustrates an image of a fringe pattern that has 
been incrementally phase shifted 90 degrees from the 
fringe pattern shown in Figure 3a. 
[0047] Figure 3c illustrates an image of a fringe pat- 
tern that has been incrementally phase shifted 90-de- 
grees with respect to the fringe pattern shown in Figure 
3b. 

[0048] Figure 3d illustrates a 90 degree incremental 
phase shift with respect to the fringe pattern shown in 
Figure 3c. 

[0049] As shown in Figure 3a, the right edge of the 
pyramid-shaped object 28 has a light bar 30, which 
slowly gets smaller in Figures 3b and 3c until it disap- 
pears in Figure 3d. 

[0050] Similarly, in Figure 3a, the peak 32 of the pyr- 
amid is initially covered by a light bar 34, which gradually 
moves to the right during the successive phases, shown 
in Figures 3b, 3c, and 3d. 

[0051] In general, individual incremental phase shift 
images like those shown in Figure 3 can be filtered for 
noise reduction and are further analyzed for conditions 
like specular reflection or holes in the part. If any spec- 
ular reflection exists, proper remedies can be taken so 
that the specular reflection is avoided in the final results. 
If holes are detected, they are recognized and marked 
as regions with no valid shape information. These re- 
gions will, therefore, be ignored in the final analysis ad- 
dressed in the disclosed method. 
[0052] The particular process and method of detect- 
ing specular reflections and holes and the application of 
proper remedies is not critical. The disclosed system 
and process, therefore, assumes, without losing gener- 
ality, that the images obtained from incremental phase 
shifts are free of noise, specular reflection, and holes. 
[0053] The three 90 degree incremental phase shifts 
can be used together with the original one to compose 
a combined phase shift image using the following for- 
mula: 
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Where: 



/ is the resulting phase shift value, 

l-i , l 2 , l 3 , and l 4 are the respective intensities for any 
pixel under consideration for each of the phase shift 
images and in this case :- 

l-i is the intensity of the image shown in Figure 

3a; 

1 2 is the intensity of the image shown in Figure 
3b; 

1 3 is the intensity of the image shown in Figure 
3c; and 

1 4 is the intensity of the image shown in Figure 
3d. 

[0054] Applying the above formula to each pixel in the 
images shown in Figures 3a through 3d results in a com- 
bined phaseshift image, which is the combination of Fig- 
ures 3a through 3d, such as the one illustrated in Figure 
4. 

[0055] Based on this result, a so-called RODP dia- 
gram of the combined phase shift image can then be 
generated for any horizontal or vertical scan of the im- 
age. 

[0056] The upper portion of Figure 4, as generally in- 
dicated by reference number 36, illustrates an example 
of such a RODP diagram. In Figure 4, the horizontal axis 
of the ROPD diagram represents the spacing of the bars 
and their vertical phase angles. 

[0057] The vertical phase angle ranges from 0 to 2U 
due to the sinusoidal nature of the intensity distribution. 
As shown in the combined phase shift image of Figure 
4, each of the vertical bars is generally consistent from 
the top of the image to the bottom of the image. Further, 
the peaks 38 of each of the bars in the ROPD diagram 
does exist for any horizontal scan and is generally 
spaced with consistently varying intervals, indicating 
that there are no occlusions. 

[0058] As discussed, the preferred sensor system 1 2 
is used to inspect the surface of an object 18. As is 
known, each sensor 12 and each fringe generator 14 
have a line-of-sight visibility, and thus visibility issues 
must betaken into account. The visibility issues fall into 
one of four categories. 

[0059] In the first category, both the SLM 14 and the 
sensor 1 2 have a line-of-sight visibility to the object sur- 
face 18. This is the necessary condition for an image to 
be useful for measurement purposes. 
[0060] In the second category, the SLM 14 has line- 
of-sight visibility to the object 1 8, but the sensor 1 2 does 
not have line-of-sight visibility. 

[0061] In the third category, the sensor 12 has line-of- 
sightvisibility, but the SLM 14does not have line-of-sight 
visibility. 

[0062] In the fourth category, neither the sensor 12 
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nor the SLM 1 4 has line-of-sight visibility. 
[0063] In accordance with the preferred system 10, 
the computer 1 6 determines which of the four categories 
the sensor 12 and the SLM 14 fall into. 
[0064] In the first category, where both the sensor 12 
and the SLM 14 have line-of-sight visibility, then all the 
bars in the phase shift image should be continuous, i.e. 
no broken bars. Another characteristic of the first cate- 
gory is that in the ROPD diagram, there is a peak cor- 
responding to each of the bars in the image regardless 
of the horizontal scan line position. The properties of an 
image that falls into the first category are illustrated in 
Figures 3a through 3d and Figure 4. 
[0065] In the second category, where the SLM 14 has 
line-of-sight visibility, but the sensor 12 does not have 
clear line-of-sight visibility, not all the bars in the phase 
shift image are continuous. Another characteristic of a 
second category image is that there will be some miss- 
ing peaks in the ROPD diagram if the horizontal scan 
line passes the region where the sensor 12 does not 
have clear line-of-sight visibility. This is illustrated in Fig- 
ures 5 and 6 and as shown in Figures 5a through 5d, 
fringe patterns are projected onto the pyramid-shaped 
object 28 in incremental 90 degree phase shifts. How- 
ever, as shown, the angle of incidence of the sensor 12 
is such that the sensor 12 does not have clear line-of- 
sight to the entire surface. 

[0066] As shown, the sensor 12 is not able to viewthe 
left most side of the pyramid. Figure 6 illustrates a com- 
bined phase shift image of the images of Figures 5a 
through 5d. As shown, not all the bars in Figure 6 are 
continuous. Further, some of the peaks in the ROPD di- 
agram 36 are missing ; as generally indicated by refer- 
ence number 40. 

[0067] The second category definition may be extend- 
ed to include cases in which the sensor does have the 
absolute line-of-sight, but is on the verge of losing it. An 
image obtained in such a case is valid in theory but dif- 
ficult to use in practice because a surface area for which 
the line-of-sight condition is almost violated usually oc- 
cupies only a small area in the overall image. The small 
area results in insufficient pixel resolution to represent 
this area. 

[0068] Such a condition is reflected in an ROPD dia- 
gram as "pinched up peaks of intensity" and thus can 
be detected. 

[0069] In the third category, where the sensor 12 has 
line-of-sight visibility, butthe SLM 1 4 does not have line- 
of-sight visibility, all the bars in the phase shift image are 
continuous. However, in this image, an occluded area 
will exhibit a significant variation of spacing between the 
bars. Another characteristic of a third category image is 
that in the ROPD diagram, there is excessively large 
spacing between peaks. Figures 7 and 8 illustrate a third 
category scenario. As shown in Figures 7a through 7d, 
fringe patterns are projected onto the pyramid-shaped 
object 28 in incremental 90 degree phase shifts. As 
shown, the sensor 12 has line-of-sight visibility to the 



entire surface of the object 28. However, the SLM 14 
does not have line-of-sight visibility as the surface of the 
object 28 does not have continuous vertical bars on the 
left side of the pyramid, as generally indicated by refer- 

5 ence number 42. 

[0070] As shown in Figure 8, there are no missing 
peaks in the ROPD diagram 36. However, the spacing 
between the peaks in the occluded area is very large, 
forming a flat plateau with an increased intensity be- 

10 tween the lowest and the highest intensity values, as 
generally indicated by the boxed area 44. 
[0071 ] The third category definition may be extended 
to include cases in which the SLM is on the verge of 
losing the line-of-sight visibility. An image obtained in 

15 such a case is valid in theory but of a poor quality in 
practice because a surface area that barely satisfies the 
line-of-sight condition will exhibit a narrow range of light 
intensity. The lack of light intensity resolution to repre- 
sent the area results in deficiency of measurement ac- 

20 curacy and thus needs to be avoided. Such a condition 
corresponds to "flatten peaks" in the ROPD diagram and 
thus can be detected. 

[0072] In the fourth category, where neither the SLM 
1 4 or the sensor 1 2 has line-of-sight visibility, the char- 
ts acteristics of both the second category and the third cat- 
egory may exist. 

[0073] It will be understood in Figures 3, 5 and 7. each 
pair of vertical bars (one in black and one in white) in an 
image represents a complete phase of the fringe pat- 
so tern, for which the lowest intensity is located somewhere 
along the middle of the black bar and the highest inten- 
sity along the middle of the white bar. The intensity in- 
between the peaks, that is to say, two adjacent highest 
intensity locations is, in fact of sinusoidal natural and 
35 varies continuously. As will be understood, the figures 
illustrate black and white bars which are representative 
of a grey-scaled fringe pattern, which cannot be repro- 
duced due to a lack of shading resolution. Moreover, the 
continuously varying nature of intensity will be obvious 
40 to those skilled in the art. 

[0074] A similar simplification is shown in Figures 4, 
6, and 8. As will be understood, the lowest intensity of 
any image in those figures should be the right edge of 
a black bar and the highest intensity should be the left 
45 edge of its right neighbour (a white bar). The actual in- 
tensity in-between is also sinusoidal in nature and will 
also be obvious to those skilled in the art. 
[0075] The representations shown in Figures 3 to 8 
are preferably of digital images or of images that can be 
50 converted into digital images. Further, the light intensity 
for any pixel in a digital image can be queried for various 
analysis purposes. 

[0076] Figure 9 schematically illustrates the operation 
of the preferred system for determining if an object un- 
55 der inspection is free of occlusions for a given sensor 
location and for a given fringe generator location. 
[0077] The characteristics of the four categories de- 
scribed in detail above can be used to assist in such a 
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procedure. I n accordance with the preferred sensor sys- 
tem 1 0, the phase shift images are preferably input into 
the computer 16, as generally indicated by reference 
number 50. 

[0078] The combined image is then computed ac- 
cording to the equation: 

/ = arcfa "(l4-l2)/ (11-13) 

and is then generated, as generally indicated by 
reference number 52. 

[0079] The resultant combined image is then 
scanned, as generally indicated by reference number 
54. The combined image is preferably scanned bar by 
bar. Thus, the combined image is usually scanned hor- 
izontally. 

[0080] The scanned image is analyzed to determine 
whether both the sensor 12 and the SLM 14 have line- 
of-sight visibility to a given scan line, as generally indi- 
cated by reference number 56. 

[0081] If the scan line meets the requirements of the 
first category then the next bar in the image is scanned, 
as generally indicated by reference number 62. If, how- 
ever, when the scanned image is analyzed at reference 
number 56, it does not meet the conditions of the first 
category, it is analyzed to determine whether it meets 
the conditions of the second category, as generally in- 
dicated by reference number 64. 

[0082] If the image meets theconditions of the second 
category then the second category flag is set to on, as 
is generally indicated by reference number 66. Afterthe 
second category flag is checked, the image is scanned 
to determine whether it meets the conditions of the third 
category, as generally indicated by reference number 
68. If the scanned image meets the conditions of the 
third category then the third category flag is set to on, 
as is generally indicated by reference number 70. 
[0083] If, however, the conditions of the second cate- 
gory are not met during image analysis at reference 
number 64 the second category flag is not set and the 
image is analyzed at reference number 68 to determine 
whether it meets the conditions of the third category. If 
the image does not meet the conditions of the third cat- 
egory, the third category flag is not set and the next por- 
tion of the image is checked to determine whether it 
meets the conditions of the first category at reference 
number 56. 

[0084] Once the entire image has been scanned then 
the computer 1 6 checks to determine whether eitherthe 
second category flag or the third category flag has been 
set to on at reference number 58. If not, then the image 
belongs to the first category, as generally indicated by 
reference number 60. If either category flag has been 
set to on , at reference number 58, then the image is cat- 
egorized in a category other than the first category, as 
generally indicated by reference number 72, and de- 
pending upon what category flag has been checked. 



[0085] If the current sensor location does not belong 
to the first category, corrective steps can be taken to 
movethesensorto afirst category location. Specifically, 
if the image belongs to the second category, then the 
5 position of the sensor 1 2 needs to be adjusted. A correct 
direction can be calculated based on the concept of 
search in the direction of the quickest reappearance of 
the missing peaks. 

[0086] The sensor 1 2 is then moved along that direc- 
tion for a small step and the result is then analyzed. Mul- 
tiple directions may have to be considered if the direc- 
tion of the quickest reappearance leads to conditions of 
the third category. This process repeats until none of the 
conditions of the second category are met and none of 
the conditions of the third category are created either in 
the course of doing so. Thefinal result is a good location 
of the sensor 1 2 that satisfies the conditions of the first 
category. 

[0087] If the image belongs to the third category, then 
the position of the SLM 14 needs to be adjusted. Simi- 
larly, a correctional direction can be determined in a way 
the plateau in the ROPD diagram disappears the quick- 
est. The SLM 14 is then moved along that direction for 
a small step. This process repeats until none of the con- 
ditions of the third category are met. Multiple directions 
may have to be considered if the direction of the quickest 
disappearing plateau leads to conditions of the second 
category. The final result is a good location of the SLM 
that satisfies the conditions of the first category. 
[0088] If the image belongs to the case of the third 
category in which the fringe generator is on the verge of 
losing the line-of-sight visibility, such as when the gra- 
dient of the object shape along the line-of-sight direction 
from the fringe generator is exceedingly large, an alter- 
native to moving the fringe generator to a better position 
can be utilized. This alternative includes adjusting the 
spacing of the fringes as they come out of the fringe gen- 
erator so that the high gradient area on the object will 
be exposed to more tightly spaced fringes to compen- 
sate for the lack of light intensity variation problem. This 
can be achieved either by adjusting the existing fringes 
or by projecting additional fringes only onto the area in 
question, in which case all the equations established for 
the general purpose of computing the combined phase 
shift image apply automatically. 
[0089] Sometimes, iterations may not yield a feasible 
location for either the sensor 1 2 or the SLM 1 4. If this 
situation occurs, the area under inspection needs to be 
split into smaller regions. This partition process contin- 
ues until, for each region, the sensor scenario belongs 
to the first category. 

[0090] The sensor system 1 0 can be used in a variety 
of applications. Most importantly, it can be used to assist 
in measuring physical parts with no matching CAD mod- 
els available. This is a frequently encountered task that 
occurs in the reverse engineering of a commercial prod- 
uct of interest. The sensor system can also be used to 
assist in die development, such as by "fingerprinting" or 
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capturing images of said fringe pattern on said sur- 
face, said sensor (12) being located at a different 
position from said fringe generator (1 4) , a computer 
(16) in communication with both said fringe gener- 
5 ator (14) and said sensor (12) to determine whether 

each of said fringe generator (14) and said sensor 
(1 2) have line-of-sight visibility to said surface to be 
measured. 

10 2. A system as claimed in claim 1 wherein said fringe 
generator includes a light source (20) directed at 
said surface and a spatial light modulator (24) dis- 
posed between said light source (20) and said sur- 
face. 

15 

3. A system as claimed in claim 1 or in claim 2 wherein 
the system further comprises a phase shifter for 
generating incremental phase shifts of said fringe 
pattern on said surface, which incremental phase 
20 shifts are captured as images by said sensor (12). 



measuring the "soft" or trial die for the "hard" or produc- 
tion die. 

[0091] This allows for the capture of knowledge and 
work, which is used to create the soft die. Further, the 
system 1 0 can be used to scan a panel taken out of a 
die. The scanned information is then compared to the 
CAD model. A colour map visualizing any discrepancy 
between the produced part and its intended shape can 
be created to assist a sophisticated die maker in inter- 
preting and analyzing the quality of the part. Additionally, 
with the process, a CAD model of a part can be shown 
on the corresponding physical part to perform part ver- 
ification. 

[0092] The sensor system 1 0 can also be utilized to 
capture production problems. For example, the head- 
lights and the headlight openings of a vehicle could be 
scanned to determine which of the two parts is causing 
interference so that the production problems can be cor- 
rected. This allows parts to be tested individually as they 
are coming down the line instead of waiting for a statis- 
tically significant sampling size, as is currently neces- 
sary. Moreover, the disclosed system 1 0 can be used to 
"fingerprint" a hard tool as well when it is originally cre- 
ated. This is important because as a hard tool is used, 
its shape can be changed. 

[0093] Thus, if a hard tool breaks later into its life, the 
fingerprint of the part at the time it broke will most likely 
not be the same as the fingerprint when the hard tool 
was originally created. This process will also allow the 
life of a hard tool to be predicted. 
[0094] Another application for the disclosed system 
1 0 is with a vendor or supplier company of parts. If the 
vendor has an analytical CAD model of the part or parts 
being made, periodic scan can be performed on the part 
during development. This process could reveal that al- 
though the part does not fall within the tolerances spec- 
ified by the manufacturer, it works and does not need to 
be modified any further. 

[0095] The system 10 could also be used to scan a 
vehicle wheel to determine if it has five nuts located ther- 
eon or not. The above applications are only illustrative 
and the disclosed system can be used in a variety of 
other applications as will be understood by one of skill 
in the art. 

[0096] While the invention has been described in 
terms of preferred embodiments, it will be understood, 
of course, that the invention is not limited thereto since 
modifications may be made by those skilled in the art, 
particularly in light of the foregoing teachings. 



Claims 

1 . A system (1 0) for measuring at least one dimension 
of a physical part (18) characterised in that the 

system comprises a fringe generator (14) for pro- 
jecting a fringe pattern of light onto a surface of the 
physical part (18) to be measured, a sensor (12) for 



4. A system as claimed in claim 3, wherein said phase 
shifter is digitally controlled by said computer (1 6). 

25 5. A system as claimed in claim 3 or in claim 4 wherein 
said computer (16) determines a combined image 
based on said captured images of said incremental 
phase shifts. 

30 6. A system as claimed in claim 5 wherein said com- 
bined image is determined according to the equa- 
tion: 

35 l = arctan((l 4 -l 2 )/(l 1 -l 3 )) 

7. A method for providing automatic sensor planning 
to measure the shape and dimension of a physical 
part (18), characterised in that the method com- 

40 prises projecting a fringe pattern on a surface of the 
physical part (18), projecting an incrementally 
phase shifted fringe pattern on said surface and de- 
termining based on said projected fringe pattern 
and said projected incrementally phase shifted 

45 fringe pattern whether a sensor (12) and a fringe 
generator (14) have a clear line-of-sight to said sur- 
face. 



8. A method as claimed in claim 7 wherein a determi- 
ne nation of whether the sensor (12) and said fringe 

generator (1 4) have a clear line-of-sight is based on 
an image of the projected fringe pattern and said 
projected incrementally phase shifted fringe pat- 
tern. 

55 

9. A method as claimed in claim 7 or in claim 8 wherein 
the method further comprises computing a com- 
bined image resulting from an image of the project- 
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ed fringe pattern and an image of the projected in- 
crementally phase shifted pattern. 

A method as claimed in claim 9 wherein the method 
further comprises scanning said combined image 5 
to determine whetherthe sensor (12) and the fringe 
generator (1 4) have a clear line-of-sight, a poor line- 
of-sight, or no line-of-sight. 

A method as claimed in claim 9 or in claim 1 0 where- io 
in the method further comprises adjusting a position 
of said sensor (1 2) if any bars in said combined im- 
age are not continuous. 

A method as claimed in claim 9 or 10 wherein the 15 
method further comprises adjusting a position of 
said fringe generator (1 4) if a significant spacing ex- 
ists between any two adjacent bars in said com- 
bined image. 

20 

A method as claimed in claim 1 0 the method further 
comprising adjusting said fringe generator (14) so 
that a surface area with poor line-of-sight is ex- 
posed to more tightly spaced fringe bars relative to 
a surface area with clear line-of-sight. 25 

A method as claimed in any of claims 7 to 1 3 where- 
in the method further comprises capturing an image 
of said projected fringe pattern and capturing an im- 
age of said projected incrementally phase shifted 30 
fringe pattern. 
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